Lecture Agenda & Discovery Roadmap

The Shift to Closed-Loop Autonomous Science

• Classical materials/drug discovery relies on manual hypothesis formulation, serial bench chemistry, and intuitive trial-and-error, which often takes 10 to 20 years from lead to deployment.
• HTS (High-Throughput Screening) vs. Adaptive Agents: Static HTS blindly tests pre-enumerated libraries, whereas closed-loop agents adapt their search policy online based on experimental telemetry.

Molecular Representations: Discrete Graphs & DAGs

Why Standard RL Fails in Chemical Design

Generative Flow Networks (GFlowNets): Core Theory

• The Sampling Objective: Train a stochastic policy $\pi(a\vert s)$ such that the terminal sampling probability is proportional to a positive reward: $$\pi(x) = \frac{R(x)}{Z} = \frac{R(x)}{\sum_{x' \in \mathcal{X}} R(x')}$$
• Network Flow Formulation: State DAG acts as a flow network: source $s_0$ enters with flow $F(s_0) = Z$; terminal sinks exit with flow $R(x)$.
• Flow Consistency: At every intermediate state $s'$, incoming flow equals outgoing flow: $$\sum_{s, a : T(s, a) = s'} F(s, a) = R(s') + \sum_{a' \in \mathcal{A}(s')} F(s', a')$$

Fig. 1 — Bengio et al. (2021), arXiv:2106.04399

GFlowNet Training: The Flow-Matching Objective

Case Study: De Novo Small-Molecule Design

• Assembly Framework: Discrete chemical fragments joined into junction tree representations. Actions select fragment additions and stem attachment atoms.
• Target Objective: High binding affinity to soluble epoxide hydrolase (sEH), scored via an MPNN proxy of AutoDock Vina binding energy.
• Mode Discovery: GFlowNets discovered $>1{,}500$ distinct high-affinity modes ($R(x) > 8$), compared to $<100$ modes for PPO and MARS baselines.
• Diversity: Average pairwise Tanimoto similarity was $0.44$ vs $0.62$ for PPO (lower indicates greater diversity).

Fig. 5 — Bengio et al. (2021), arXiv:2106.04399

The Synthesizability Barrier in Generative Design

• Generative models readily produce billions of theoretical chemical graphs exhibiting optimal binding metrics, yet upwards of $99\%$ cannot be physically synthesized in a laboratory.
• Hidden Physical Constraints: Unstable chemical bonds, steric strains, and inaccessible stereocenters are completely invisible to naive graph rewards. Production is strictly bounded by the availability of stable, commercial precursors.
• Forward vs. Retrosynthetic Search: Forward prediction computes product $C$ from known reactants $A + B$. Retrosynthesis inverts this challenge: recursively breaking down target $C \Rightarrow B \Rightarrow A$ until reaching catalog building blocks $\mathcal{B}$.

Retrosynthesis Formalized as an MDP

• State Space $\mathcal{S}$: Precursor sets $\{m_1, \dots, m_k\}$ initiated at root $s_0 = \{m_{\text{target}}\}$. Actions $a$ denote chemical disconnection rules applied to a molecule.
• Terminal Conditions & Catalog $\mathcal{B}$: A state is solved when every molecule belongs to commercial building blocks $\mathcal{B}$. Branches fail if depth $> L_{\max} = 25$.
• Reward Structure: $$\mathcal{R}(s) = \begin{cases} +10 & \text{all } m_i \in \mathcal{B} \text{ (solved)} \\ \text{fraction solved} & \text{partial leaf rollout} \\ -1 & \text{failed branch / depth limit} \end{cases}$$

Fig. 1 — Segler et al., Nature 555, 604–610 (2018)

The 3N-MCTS Architecture (Segler et al., 2018)

• Three Integrated Neural Networks: (1) Expansion Policy prioritizing candidate rules from 301,671 templates; (2) In-Scope Filter verifying chemical/regiochemical viability; (3) Rollout Policy predicting reachability.
• Scale: Trained on $12.4\text{ million}$ single-step chemical reactions mined from Reaxys.

Fig. 2a — Segler et al., Nature 555, 604–610 (2018)

Expansion Policy & In-Scope Classification

• Branching Pruning: Exhaustive rules yield 46,175 branches per molecule. ECFP4 fingerprint expansion prunes to the top-50 actions, capturing $72.5\%$ accuracy while slashing search width.
• In-Scope Filter: Dual-branch cosine proximity network trained against $100\text{ million}$ synthetic negative reactions. Logits correlate with DFT LUMO energies ($r^2 = 0.74$) and Hammett constants ($r^2 = 0.78$).

Fig. 2b — Segler et al., Nature 555, 604–610 (2018)

MCTS Phases for Synthesis Planning

Empirical Validation: Double-Blind Expert Testing

• Quantitative Search Benchmark: Evaluated on $497$ diverse drug-like targets reported strictly post-2015. 3N-MCTS resolved $95.2\%$ of molecules in an average of $13.0\text{ s}$, while heuristic BFS solved only $45.6\%$ and required $433.4\text{ s}$.
• Human Evaluation (Test A — MCTS vs. Literature): In double-blind tests with 45 professional chemists across Münster and Shanghai, researchers exhibited no statistically significant preference ($P = 0.26$) for human literature routes ($43.0\%$) over MCTS routes ($57.0\%$).
• Human Evaluation (Test B — MCTS vs. Heuristic BFS): Chemists decisively favored 3N-MCTS ($68.2\%$ preference, $P = 0.01$), identifying heuristic BFS steps as chemically unviable.

Self-Driving Labs (from Code 💻 to Atoms ⚛️)

• A validated retrosynthetic plan remains theoretical. Physical synthesis demands automating reagent dispensing, phase extraction, heating, and characterization without manual intervention.
• Self-Driving Laboratory Architectures: Integrated robotic workcells couple liquid handling platforms with analytical instruments (HPLC, GC-MS, UV-Vis, NMR) to close the physical discovery cycle.
• Software Design Requirements: Middleware must decouple high-level agent policies from vendor-specific robotic firmware, providing asynchronous execution and automated recovery for physical faults (e.g., pipette drop errors, fluid clogs).

ChemOS: Distributed Lab Orchestration (Roch et al., 2020)

• Modular 6-Tier Architecture: Designed to coordinate closed-loop experimentation across distributed hardware platforms:
  • Workflow Manager: Asynchronous task queue & state recovery.
  • AI / Optimization: Hosts Bayesian optimization packages (Phoenics, SMAC).
  • Robotics & Characterization: Translates recipe vectors into pump/arm signals and ingests sensor telemetry.
• Decoupling Advantage: Swapping a liquid handler or optimizer requires only editing a config file.

Fig. 1 — Roch et al., PLOS ONE 15(4), e0229862 (2020)

Autonomous Process Optimization in ChemOS

• Multi-Variable Closed-Loop Tuning: Autonomous agents tuned complex continuous physical search spaces (RGB color matching, target pH, and fluid density formulations) to target specs in $\le 25$ iterations without human calibration.
• Intercontinental Hardware Control: Master ChemOS orchestrators in Cambridge, USA remotely controlled a North Robotics liquid handler and Agilent HPLC located in Vancouver, Canada.
• Robust Chromatography Calibration: Over 100 unattended consecutive runs, Bayesian optimization mapped non-linear fluidic delay parameters, achieving maximal peak separation far more efficiently than classical Design of Experiments.

Live Demo: ChemOS Active Reaction Optimization

An autonomous agent navigating an unknown continuous reaction landscape (e.g., catalyst loading vs. temperature) to maximize yield under tight wet-lab assay budgets.

Illustrative simulation only: these controls generate synthetic data. No chemicals, instruments, or wet-lab experiments are being run.

Coscientist: LLM-Driven Autonomous Chemistry

• LLMs as Laboratory Orchestrators: Frontier models exhibit broad chemical reasoning, but require tool augmentation to execute code, search scientific literature, and drive robotics.
• Modular Architecture (Boiko et al., Nature 2023): Central GPT-4 Planner issues structured ReAct commands (e.g. GOOGLE, PYTHON, DOCUMENTATION, EXPERIMENT) to dedicated sub-modules.

Fig. 1a — Boiko et al., Nature 624, 570–578 (2023)

Grounding Code in Lab Hardware: Opentrons & Cloud Labs

• Vector Search over Evolving Hardware APIs: When tasked with operating the Opentrons OT-2 heater-shaker (released post-cutoff), Coscientist queried its documentation vector index, caught syntax errors, and self-corrected method calls dynamically.
• Cloud Lab Execution: Compiled procedures into Emerald Cloud Lab's Symbolic Lab Language (SLL), selecting correct parameters across 114 instruments and 1,110 reagents.

Fig. 3 — Boiko et al., Nature 624, 570–578 (2023)

Physical Validation: Catalytic Cross-Coupling Reactions

• Autonomous Benchmarking: Coscientist was tasked with synthesizing complex biaryl compounds via palladium-catalyzed Suzuki-Miyaura and Sonogashira cross-coupling reactions.
• End-to-End Decision Execution:
  • Reagent Pairing: Selected compatible reaction partners without human prompting.
  • Stoichiometry: Executed isolated Python code to determine exact dispensing volumes.
  • Thermal Control: Scheduled Opentrons liquid transfers and heated at $75^\circ\text{C}$ for 60 min.
• Analytical Confirmation: Reaction mixtures were characterized via Gas Chromatography-Mass Spectrometry (GC-MS), confirming target product formation matching pure analytical standards.

Empirical Execution: Palladium Catalysis & GC-MS Validation

Fig. 5 — Boiko et al., Nature 624, 570–578 (2023). TIC chromatograms confirm biphenyl (9.53 min) and diphenylacetylene (12.92 min).

Frontiers, Guardrails, and Project Horizons